1. Field of the Invention
The present invention relates to machine learning and pattern recognition, and in particular, relates to a method and a device for learning of a classifier.
2. Description of the Related Art
In the field of object detection and tracking, a one-class classification method was extensively employed in last few decades. As described in Moya, M. and Hush, D. (1996). “Network constraints and multi-objective optimization for one-class classification”. Neural Networks, 9(3):463-474. doi: 10.1016/0893-6080(95)00120-4, in the one-class classification method, it tries to distinguish one class of objects from all other possible objects, by learning from a training set containing only the objects of that class.
The support vector data description (SVDD) method is a powerful kernel method for the one-class classification. As described in D. Tax and R. Duin, “Support vector domain description”. Pattern Recognit. Lett., vol. 20, pp. 1191-1199, November, 1999, the SVDD method aims at finding a minimum-volume sphere (also referred to as a minimum sphere), such that all or most of the target training samples are enclosed by the sphere. FIG. 1 shows a schematic diagram of the SVDD method. The points on the minimum sphere are referred to as support vectors, which characterize the minimum sphere for enclosing the training samples.
Because of the good learning capacity and the generalization performance of the SVDD method, success of SVDD has recently been shown in various applications. The performance and the accuracy of the SVDD method rely on the availability of a representative dataset of training samples. However, in some online applications such as video surveillance and image retrieval, the SVDD method may fail because there are not enough training samples in the initial state.
In order to address above issues, online learning techniques are introduced to the art. In the online learning techniques, a classifier is learned by one or few sample(s) firstly, and then updated along with the procedure of system executing. The online learning techniques achieve a great success because of the excellent adaptivity thereof and the limited memory requirement thereof. An Online SVDD method and an Incremental SVDD method are the techniques which are widely used in real-time object detection currently, both of which are the online learning methods based on the support vector methods.
In D. M. J. Tax and P. Laskov, “Online SVM learning: from classification to data description and back,” In C. et al. Molina, editor, Proc. NNSP, 2003, pp. 499-508, the Online SVDD method is described. FIG. 2A shows a simplified flowchart of the Online SVDD method. Firstly, an original classifier based on SVDD and newly added labeled samples (i.e. positive samples) are obtained. The so-called positive samples are the samples which have been determine to belong to a target category. Then, the relatedness of every sample is calculated with the classifier. Next, the most irrelevant samples are selected from the previously training sample set for the original classifier. Then, new samples are added and the most irrelevant samples are removed based on the newly added positive samples. Lastly, the classifier is relearned using remained samples and new added samples.
FIG. 2B shows a schematic diagram of the Online SVDD method. As shown in FIG. 2B, in the updating process of the classifier, the samples in the original sample set are replaced by the newly added samples, and the hypersphere of the classifier is increasingly enlarged in the updating process.
In Xiaopeng Hua, Shifei Ding, “Incremental Learning Algorithm for Support Vector Data Description”, JOURNAL OF SOFTWARE, VOL. 6, NO. 7, July 2011, the Incremental SVDD method is described, in which the SVDD incremental learning is performed by analyzing the possible changes of support vector set (SVs) after new samples are added to training sample set. FIG. 3A shows a simplified flowchart of the Incremental SVDD method. Firstly, an original classifier based on SVDD and newly added labeled samples are obtained. Then the previously training set is partitioned into support vector set and non support vector set. Next, whether there are new added samples outside the hypersphere of the original classifier is verified; if there are, these samples will be added into training sample set. Then, the samples which are most likely to be new support vectors are found in the non support vector set. Lastly, the classifier is relearned using the samples outside the hypersphere of the original classifier, the samples which are most likely to be new support vectors in the non support vector set and the support vector set.
FIG. 3B shows a schematic diagram of the Incremental SVDD method. As shown in FIG. 3B, in the updating process of the classifier, the classifier is updated based on the samples outside the hypersphere of the original classifier, the samples which are most likely to be new support vectors in the non support vector set and the original support vector set, and the hypersphere of the classifier is increasingly enlarged in the updating process.
As can be learned from FIGS. 2B and 3B, when the diversity of training samples is small, the learned hypersphere is small, that is, when the classifier is trained by a small set of training samples, the threshold of the learned classifier is small. When the diversity of training samples is enlarged, the learned hypersphere is enlarged too, that is, when the new samples are added, the threshold of learned classifier is increased.
That is, the hypersphere will be enlarged when the number of training samples increases, so the performance of learned classifier will be unstable. In addition, if some newly added samples are falsely labeled (it always happens in determination by classifier automatically), the performance of the learned classifier will become worse and worse. So the Online SVDD method and the Incremental SVDD method require the samples being labeled correctly (as positive or negative) in advance, in order to keep the classifier's performance to be stable over long running time.
Accordingly, there is a need for a novel technique to address any problem in the prior art.